Using Plan Recognition for Interpreting Referring Expressions

Referring expressions such as “a long meeting” and “a restaurant near my brother’s” depend on information from the context to be accurately resolved. Interpreting these expressions requires pragmatic inferences that go beyond what the speaker said to what she meant; and to do this one must consider the speaker’s decisions with respect to her initial belief state and the alternative linguistic options she may have had. Modeling reference generation as a planning problem, where actions correspond to words that change a belief state, suggests that interpretation can also be viewed as recognizing beliefstate plans that contain implicit actions. In this paper, we describe how planners can be adapted and used to interpret uncertain referring expressions. Recognizing and Synthesizing Phrase-Level Plans Although plan recognition has a long connection with natural language processing (NLP), historically researchers have focused on two high-level problems: (1) inferring a speaker’s communicational and task goals from a speech act (Allen and Perrault 1980; Litman and Allen 1984; Hußmann and Genzmann 1990) and (2) identifying a plan embedded in the content of text (Bruce 1977; Charniak and Goldman 1993; Raghavan and Mooney 2011). In this paper, we examine plan recognition at a finer grained level of linguistic analysis, where observed actions correspond to individual words1 and the plans correspond to referring expressions. Researchers in the NLP sub-field of natural language generation (NLG) have deployed AI planning techniques to the problem of sentence planning (Koller and Stone 2007; Bauer 2009; Koller, Gargett, and Garoufi 2010; Garoufi and Koller 2010). These planning approaches treat the lexicon as the domain theory from automated planning: each action ∗We are grateful for the support of the sponsors of the MIT Media Lab. Copyright © 2013, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. There is no good reason to requirewords to be the primitive actions. A more general theory would include all lexical units: morphemes to words to idiomatic phrases. Here, we describe our actions as “words” for lack of a better lexical unit. (word) is annotated with a description of its meaning (implementing a theory of lexical semantics) including the language’s serialization constraints, expressed by a lexicalized grammar theory. The planner’s objective is to choose a sequence of words that adhere to syntactic, semantic and pragmatic constraints in order to achieve the communicational goals while minimizing costs. In this paper, we describe the problems of generating and interpreting referring expressions, and formulate them as planning and plan-recognition problems. Next, we review two cost-difference approaches to plan recognition and investigate where they can contribute to interpreting referring expressions. What is plan recognition? Imagine you see a robot at position s2, equidistant from two goal states: g1 and g2 (Figure 1). Without additional knowledge, you must concede that both goals are equally likely. However, if you are also given (or estimate) the robot’s initial state, s0, and some observed actions O = 〈a1, a2〉, then you can reason about the possible decisions it made to get from s0 to s2. This may give you more information: that the robot’s intention was more likely g2, which you may have inferred by reasoning along the lines “if it had wanted to go to g1, it would have taken a left sooner than at s1.”